Visual image sensor organ replacement: implementation

ABSTRACT

Method and system for enhancing or extending visual representation of a selected region of a visual image, where visual representation is interfered with or distorted, by supplementing a visual signal with at least one audio signal having one or more audio signal parameters that represent one or more visual image parameters, such as vertical and/or horizontal location of the region; region brightness; dominant wavelength range of the region; change in a parameter value that characterizes the visual image, with respect to a reference parameter value; and time rate of change in a parameter value that characterizes the visual image. Region dimensions can be changed to emphasize change with time of a visual image parameter.

RELATED APPLICATION

This application is a continuation-in-part of a patent applicationentitled “Visual Image Sensor Organ Replacement,” U.S. Ser. No.11/239,450, filed 23 Sep. 2005.

ORIGIN OF THE INVENTION

This invention was made, in part, by one or more employees of the U.S.government. The U.S. government has the right to make, use and/or sellthe invention described herein without payment of compensation,including but not limited to payment of royalties.

FIELD OF THE INVENTION

This invention relates to implementation of use of audio signalparameters as representatives of time varying or constant visual signalparameter values.

BACKGROUND OF THE INVENTION

Present development of fast, cheap and miniaturized electronics andsensory devices opens new pathways for the development of sophisticatedequipment to overcome limitations of the human senses. Humans relyheavily on vision to sense the environment in order to achieve a widevariety of goals. However, visual sensing is generally available onlyfor a limited visible range of wavelengths, roughly 400 nm (nanometers)to 720 nm, which is a small fraction of the range of wavelengths (180 nmthrough about 10,000 nm) at which interesting physical effects and/orchemical effects occur. Audible sensing, over an estimated audible rangeof 200 Hz (Hertz)-20,000 Hz, is similarly limited, but this range is alarger fraction of the audibly interesting range 10 Hz-10⁵ Hz). Further,use of binaural hearing to provide audible clues as to depth andrelative location is generally better developed than are thecorresponding mechanisms associated with formation of visible images.

Since the time of Aristotle (384-322 BC), humans have been interested inperceiving what is beyond normal “vision”. Roentgen's discovery ofX-Rays enabled him to see inside living tissue, and “vision” was therebyextended beyond the naked eye. In the following years, imaging andsensing techniques have developed so rapidly that astronomy, medicineand geology are just few of the areas where sensing beyond the normalvisual spectrum has been found useful. Altering and extending human“vision” changes our perception of the world

According to some recent research in evolution of the sight system foranimals, reported in “What Birds See” by Timothy H. Goldsmith,Scientific American, July 2006, pp. 68-75, certain bird species have atetra-chromatic color sensing system, with color bands spanning thenear-ultraviolet, violet, green and red wavelengths, in contrast to thetri-chromatic (for primates, humns and some birds) or bi-chromatic (forother animals) color sensing systems that cover only two or threevisible wavelength bands. The tetra-chromatic color sensing system ofthe birds allows more subtle sensing of color differences, much as HDradio claims to allow receipt of radio frequencies between the 0.2 kHzsignposts of conventional commercial radio. This extra color sensingsubtlety available to some birds is not available, and is not likely tobecome available, generally to humans and/or primates.

Further, the human audible sensing system is capable of learning toprocess and interpret extremely complicated and rapidly changingauditory patterns, such as speech or music in a noisy environment. Theavailable effective bandwidth, on the order of 20 kHz, may support achannel capacity of several thousand bits per second. The knowncapabilities of the human hearing system to learn and understandcomplicated auditory patterns provide a basic motivation for developinga visual image-to-sound mapping system.

What is needed is a system that converts “visual signals”, definedherein as signals with at least one associated wavelength in theultraviolet, the visible and/or the infrared, to one or more audiblyperceptible signals with associated audio parameters that can berecognized and distinguished by the human ear. Preferably, these signalsshould include an audible indication of change, or change rate withtime, of one or more visual image parameters. Preferably, these audiosignals should provide monaural and/or binaural signaling that isanalogous to depth clues and/or distance clues provided by visuallyperceptible images. Preferably, the audible signal parameters shouldhave an intuitive connection with the visual signal parameters to whichthe audible signal parameters correspond.

SUMMARY OF THE INVENTION

These needs are met by the invention, which provides a mapping orassociation between signals representing a selected region of a receivedvisual image and audibly perceptible signals in which M visual signalparameter values (M=1-8) are mapped one-to-one onto a selected set ofdistinguishable audible signal parameters. External multi-spectralsensors signals are translated into audible signals targeting the samehuman visual field.

The visual signal is received and one or more visual signal parametersare measured or otherwise provided, including but not limited todistinction between visual signal wavelengths in the ultraviolet, thevisible, the near-infrared and the mid-infrared. The audible signalparameter values provided as output include one or more of: an envelopefrequency f_(e); a time rate of change of the envelope frequency(analogous to “chirping” or to a Doppler effect); a carrier frequencyf_(c); an envelope frequency phase φ_(e) at a selected time, t=t_(ph,e);a carrier frequency phase f_(c) at the selected time, t=t_(ph,c); abaseline function b(t) the defines a baseline curve BB; a time rate ofchange db/dt of the baseline function; a non-undulatory, but possiblytime varying, signal amplitude a(t), measured relative to the baselinecurve BB; and a time interval (duration) Δt for the signal. The humanear may be able to distinguish the phase difference, Δφ=φ_(e)−φ_(c), butneed not recognize the absolute phases, φ_(e) and/or φ_(e).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 graphically illustrates several audio signal parameters that canbe used in the invention to provide a visual-audio association ofparameters.

FIG. 2 schematically illustrates (partial) representation of a sequenceof regions of a visual image.

FIG. 3 schematically illustrates a mapping device used to transformvisual image region parameters to audibly perceptible signal parameters.

FIG. 4 illustrates a suitable receiver/processor used in FIG. 3.

FIG. 5 is a flow chart illustrating practice of an embodiment of theinvention.

FIG. 6 illustrates an application of the system in a battlefieldsituation.

FIG. 7 graphically illustrates variation of estimated projectile impacteffective distance d(E), for death or serious injury to a nearbycombatant or disablement of an equipment item, as the projectileexplosive load E varies.

FIGS. 8A, 8B, 9A and 9B graphically illustrate frequencies andintensities of audible signals used in different versions of aprojectile impact example.

FIG. 10 schematically illustrates determination of present location of aprojectile, using observations from two spaced apart observation sites.

FIG. 11 schematically illustrates distinction between trajectories oftwo different projectiles.

DESCRIPTION OF BEST MODES OF THE INVENTION

FIG. 1 graphically illustrates M signal parameter values (M=1-8) thatcan be used to collectively characterize an undulating, audiblyperceptible signal s_(a)(t) having a single information-bearing(envelope) frequency and a single carrier frequency. This signal can becharacterized by: an envelope frequency f_(e) and corresponding timerate of change of the envelope frequency df_(e)/dt (analogous to“chirping” or to a Doppler shift); a carrier frequency f_(c); anenvelope frequency phase φ_(e) at a selected time, t=t_(p,eh); a carrierfrequency phase f_(c) at the selected time, t=t_(ph,c); a baselinefunction amplitude b(t), defining a baseline curve BB, and correspondingtime rate of change of base line amplitude db/dt; a non-undulatorysignal amplitude a(t), measured relative to the baseline curve BB; and atime interval (duration) Δt for the signal. The human ear may be able todistinguish the phase difference, Δφ=φ_(e)−φ_(c), but cannot distinguishthe absolute phases, φ_(e) and/or φ_(e). An audible signal equationincorporating all these features isS _(a)(t)=b(t)+a(t)·sin {f _(e)(t)t+φ _(e)}·sin {f _(c) t+φ _(c)}  (1)

The maximum number of parameters for the signal shown in FIG. 1 that maybe distinguished by the human ear is M=6-8, if the (absolute) selectedtime, t=t_(ph), and the absolute phases are not included. These M signalparameters may be used to audibly represent a corresponding visualregion of an image, such as vertical and horizontal coordinate ranges(versus time) of the visual region (relative to a fixed two-dimensionalor three-dimensional system), estimated distance s(t) and/or rate ofchange of distance ds/dt to a selected center of the region, regionbrightness, overall region brightness, and region predominant hue(color) or wavelength. Optionally, these audible signal parameters canbe presented simultaneously or sequentially, for any correspondingvisual image region that is so represented. In a sequentialpresentation, one or more additional audible signal parameters may beincluded, if the information corresponding to the additional parametervalue(s) is necessary for adequate representation of the image region.

The visual image may be decomposed into a sequence of K selected visualimage regions R_(k), (k=1, . . . , K, with K≧1; contiguous ornon-contiguous; overlapping or nonoverlapping) that make up part or allof the total visual image, for example as illustrated in FIG. 2. Thesequence of regions R_(k), and the corresponding sequence of audiblesignal parameters, need not exhaust the set of all regions that togethermake up the visual image. Preferably, the image regions are chosenaccording to which regions are of most interest. For example, when animage has a single image region (less than the entire image) where oneor more image parameters is changing substantially with time, thisregion may be a primary focus; and if this region slowly changes itslocation or its physical extent within the total image, the location andbreadth of this image region should correspondingly change with time.That is, the horizontal and vertical bounds and/or the center of animage region may move with time within the total image.

If the visual image changes between one time to a subsequent time, theaudible parameters representing each selected region R_(k) may alsochange with time, in a sequential manner. FIG. 1 graphically illustratessignal parameters corresponding to a mapping that can be implemented torepresent a group of visual signal parameters, representing a selectedregion R_(k) of the total image, by an audibly perceptible signal orsignals.

In one approach, a visual image region R_(k) is selected and optionallyisolated, and the corresponding audibly perceptible signal parametersare presented (1) sequentially within a time interval of selected length(e.g., 5-30 sec) or (2) as part of a single audible signal thatincorporates two or more selected audible signal parameter values.

If an audible signal parameter changes with time, continuously ordiscretely, this change can be presented according to several options:(i) change the audible parameter value continuously at a rate thatcorresponds to the time rate of change of the corresponding visualparameter value; (ii) change the audible parameter value discretely at arate corresponding to a discrete time rate of change of the visualparameter value; and (iii) change the audible parameter valuediscretely, by a selected amount, only when the magnitude of thedifference between a first value and a second value of the parameter isat least equal to a threshold magnitude, which will vary with the natureof the visual parameter.

Humans and primates rely heavily on tri-chromatic vision to sense andreact to the environment in order to achieve various goals. By contrast,other animals rely heavily, but not exclusively, on smell (e.g.,rodents), on sound (e.g., some birds), or on tetra-chromatic vision(other birds). The invention augments or replaces a human sensory visualsystem, which is deficient in many respects, with one or more auditorysignals, in order to achieve the following.

(1) Provide a capacity to sense beyond the human visible light range ofthe electromagnetic spectrum).

(2) Increase capacity of human sensing resolution, beyond the number ofrods and cones in the human eye (approximately 120 million rods and 6million color sensing cones) that limit the resolution of the images,particularly because humans rely on the subset of cones located in thefovea, which provide humans the highest visual acuity of approximately 1min of arc resolution within a field of view less than 12 degreeshorizontal by 4 degrees vertical in humans.

(3) Provide wider angle equivalent of visual sensory perception, wherethe shape and location of human eyes limit the effective human field ofview to about 200 degrees horizontally by about 150 degrees vertically.

(4) Improve the ability of a human to sense distances, which ispresently relatively poor and can be confounded by a wide variety ofvisual cues.

(5) Allow compensation for movement by the human or changes in thescene; for example, motion smear or blur can make it difficult toresolve images at resolutions achievable when the perspective of animage is not moving or changing.

(6) Allow splitting of user attention (multi-tasking using two or moresenses), where a visual image limits the range of other activities thata person can do simultaneously, such as monitoring gauges and readingtext concurrently.

(7) Provide audibly perceptible changes in an audible parameter valuethat correspond to changes, continuous or discrete, in a visualparameter value that are too small or subtle for a human eye to sense orrespond to.

(8) Provide an audible parameter value that changes in an audiblyperceptible manner only when the corresponding visual parameter changesby at least a threshold amount, and the threshold is selectableaccording to the environment.

Using the invention, a wide variety of tasks that are difficult orcumbersome to accomplish, using primarily visual indicia, can be met,including the following:

-   -   Enabling the user to substantially simultaneously focus        attention on multiple aspects of a visual field, or its audible        field equivalent;    -   Embedded human sensing of aircraft performance; and    -   Audibly indicating micro-fractures and/or thermal distortion in        materials.

In order to increase the visual image resolution obtainable via anauditory representation, a mapping is performed to distribute an imagein time. Three-dimensional spatial brightness and multi-spectral maps ofa sensed image are processed using real-time image processing techniques(e.g., histogram normalization) and are transformed into one or moretwo-dimensional maps of an audio signal as a function of frequency andof time.

The invention uses a Visual Instrument Sensory Organ Replacement (VISOR)system to augment the human visual system by exploiting the improvedcapabilities of the human auditory system. The human brain is farsuperior to most existing computer systems in rapidly extractingrelevant information from blurred, noisy, and redundant images. Thissuggests that the available auditory bandwidth is not yet exploited inan optimal way. Although image processing techniques can manipulate,condense and focus the information (e.g., using Fourier Transforms),keeping the mapping as direct and simple as possible may also reduce therisk of accidentally filtering out important clues. Even a perfect,non-redundant sound representation is subject to loss of relevantinformation in a non-perfect human hearing system. Also, a complicated,non-redundant visual image-to-audible image mapping may well be moredifficult to learn and comprehend than a straightforward visual mapping,while the mapping system would increase in complexity and cost.

FIG. 3 schematically illustrates a mapping device used to transformselected visual image parameters associated with a region to an audiblesignal with audibly perceptible parameters. One or more visual imageregion (“VIR”), representations is received and analyzed by a firststage signal receiver-processor (“R/P”) 31-1. The first stage R/P 31-1analyzes a received VIR and provides one or more (preferably as many aspossible) of the following visual signal characterization parameters ina second stage R/P 31-2: vertical and horizontal coordinate ranges ofthe region and/or its center; optional adjustment in size of regionviewed; region predominant hue (color) or wavelength; region averagebrightness and region peak brightness, using a region locator and sizingmechanism 32, a region predominant (or average) color sensing mechanism33 and a region brightness sensing mechanism 34. Output signals from thelocator mechanism 32, from the color mechanism 33 and from thebrightness mechanism 34 are received by a third stage R/P 31-3, whichprovides a collection of audible signal parameters, including time rateof change TRC of at least one parameter value.

As an example: the predominant or average color output signal from theregion color sensing mechanism 33 can be used to determine the envelopefrequency f_(e); the region brightness output signal from the regionbrightness mechanism 34 can be used to determine the envelope relativeamplitude, a₀ (constant) or a(t); the vertical and horizontal locationoutput signals from the locator mechanism 32 can be used to determinetime duration Δt (if the visual image region locations are indexed by aone-dimensional index), or to determine time duration Δt, envelopefrequency f_(e) (if the visual image region locations are indexed usinga two-dimensional index) or change rate, db/dt or df_(e)/dt. The fourvisual signal parameters can be assigned to four of six audiblyperceptible signal parameters (FIG. 1) in (⁶ ₄)=(6·5·4·3)/(4·3·2·1)=15distinguishable ways. More generally, N visual signal parameters can beassigned to M (≧N) audibly perceptible signal parameters in (^(M) _(N))distinguishable ways.

Where the time rate of change option (i) is used for a visual signalparameter value r, one can form an approximating second degreepolynomialr(t;app)={r(t _(p)){(t−t _(p+1))(t−t _(p+2))(t _(p+2) −t _(p+1))+r(t_(p+1)){(t−t _(p))(t−t _(p+2))(t _(p+2) −t _(p))+r(t _(p+2)){(t−t_(p))(t−t _(p+1))(t _(p+1) −t _(p))}/Π(t _(p) ;t _(p+1) ;t _(p+2)),  (2)Π(t _(p) ;t _(p+1) ;t _(p+2))=(t _(p+2) −t _(p+1))(t _(p+1) −t _(p))(t_(p+2) −t _(p)),  (3)(t₁<t₂< . . . <t_(p)<t_(p+1)<t_(p+2)< . . . ) and compute r(t) and dr/dtusing the approximating polynomial r(t; app) and dr(t; app)/dt,respectively. Approximating polynomials of degree higher than two canalso be used here.

Where the time rate of change option (ii) is used for a visual signalparameter value r, a sequence of ratiosv2(t _(p) ;t _(p+1))={r(t _(p+1))−r(t _(p))}/(t _(p+1) −t _(p)),  (4)is computed for the sequence of times {t_(p)}.

Where the time rate of change option (iii) is used for a visual signalparameter value r, the time rate of change ratios of interest become

$\begin{matrix}\begin{matrix}{\;{{v\; 3\left( {t_{p};t_{p + P}} \right)} = {0\mspace{14mu}\left( {{{{{if}{{{r\left( t_{p} \right)} - {r\left( t_{p + q} \right)}}}} < {\Delta\;{r({thr})}{\mspace{11mu}\mspace{11mu}}{for}\mspace{14mu} q}} = 1},2,\ldots\mspace{14mu},{P - p}} \right)}}} \\{= {{\left\{ {{r\left( t_{p + P} \right)} - {r\left( t_{p} \right)}} \right\}/\left( {t_{p + P} - t_{p}} \right)}\left( {{{if}{{{r\left( t_{p} \right)} - {r\left( t_{p + q} \right)}}}} \geq {\Delta\;{r({thr})}}} \right.}} \\{{{{for}\mspace{14mu} q} = 1},2,\ldots\mspace{14mu},{{P - p - {1\mspace{14mu}{and}\mspace{14mu}{{{r\left( t_{p} \right)} - {r\left( t_{p + P} \right)}}}}} \geq {\Delta\;{{r({thr})}.}}}}\end{matrix} & (5)\end{matrix}$

The analysis performed by each of the mechanisms 32, 33 and 34 is notinstantaneous, and the associated time delays may not be the same foreach analyzer. For this reason, an overall time delayΔt(o1)≧min{Δt(location,size),Δt(color),Δt(brightness)}  (6)is preferably imposed, using a time delay mechanism 35, before anaudible signal (or audible signal sequence) incorporating the 1 throughM=M1+M2 audible signal parameters is audibly displayed, where M2 is thenumber of parameter values that can change with time and M1 is thenumber of remaining parameters. If the audible signal parameters aredisplayed sequentially, mot simultaneously or collectively, this timedelay might be reduced or eliminated. The overall time delay isimplemented by a time delay mechanism 35, which incorporates anappropriate time delay value for each of the audible signal parametersreceived from the first stage R/P 31-1. An audible signal formationmechanism 36 (optional) forms and issues either: (1) an audiblyperceptible, ordered sequence of the set of M audible signal componentsASC(m), m=1, . . . , M, (or a subset thereof), or (2) a collectiveaudibly perceptible signal APS incorporating the set (or a subset) ofthe audible signal components. The output signal from the audible signalformation mechanism 36 is perceived by a human or other animalrecipient.

The R/P 40, illustrated in FIG. 4, includes one or more of thefollowing; a carrier/envelope frequency (f_(c)) analyzer 41; an envelopeamplitude analyzer 42; an envelope-carrier frequency phase difference(Δφ) analyzer 43 and baseline function (b(t)) analyzer 44 that estimatethe phase difference at a selected time and determines the baselinefunction and baseline time rate of change; and a relative signalamplitude (a₀ or a(t)) analyzer 45, relative to the baseline function ata corresponding time.

The analysis performed by each of these analyzers is not instantaneous,and the associated time delays may not be the same for each analyzer.For this reason, an overall time delayΔt(o2)≧min{Δt(f _(e)),Δt(f _(c)),Δt(Δφ),Δt(b),Δt(a)}  (7)is preferably imposed before an audible signal incorporating the Mconverted visual signal parameters is audibly displayed. If theconverted visual signal parameters are audibly displayed sequentially,rather than simultaneously, this time delay might be reduced oreliminated. The overall time delay is implemented by a time delaymechanism 46, which incorporates an appropriate time delay for each ofthe audible parameters received from the R/P 31. A signal formationmodule 47 forms a composite audible signal representing an audible imagecomponent, and issues this component as an output signal.

Where the amplitude a(f) is constant, the signal shown in FIG. 1 may berepresented in an alternative form

$\begin{matrix}\begin{matrix}{\left. {F_{VIR} = {{b(t)} + {a_{0}\sin\left\{ {{f_{e}\left( {t - t_{\phi}} \right)} + {\Delta\phi}} \right)}}} \right\}\sin\left\{ {f_{e}\left( {t - t_{\phi}} \right)} \right\}} \\{= {{b(t)} + {a_{0}\left\{ {{\cos\left\{ {{\left( {f_{c} - f_{e}} \right)\left( {t - t_{\phi}} \right)} - {\Delta\phi}} \right\}} - {a_{0}\left\{ {\cos\left\{ \left( {f_{c} + f_{e}} \right) \right.} \right.}} \right.}}} \\\left. {\left( {t - t_{\phi}} \right) + {\Delta\phi}} \right\}\end{matrix} & (8)\end{matrix}$

The carrier/envelope frequency analyzer 42 forms a sequence ofcorrelation signals, computed over a time interval of length T,C1(f _(cs))=(1/T)∫F _(VIR)(t)sin {f _(cs) t)dt,  (9A)C2(f _(cs))=(1/T)∫F _(VIR)(t)cos {f _(cs) t)dt,  (9B)at each of a spaced apart sequence of “translated” carrier frequencies,f_(c1) in a selected carrier frequency range, f_(c1)≦f_(cs)≦f_(c2),where f_(cs) is not yet known, and provides an estimate of two spacedapart frequencies f_(cs1)=f_(c)+f_(e) and f_(cs2)=f_(c)−f_(e),associated with the VIR, where the correlation combination, C1²+C2², hasthe highest magnitudes. The envelope and carrier frequencies are thenestimated fromf _(c)=(f _(cs1) +f _(cs2))/2,  (10A)f _(e)=(f _(cs1) −f _(cs2))/2.  (10B)

The envelope-carrier phase difference Δφ and relative amplitude a₀ aredetermined by computing the correlations(1/T)∫F _(VIR)(t)sin {f _(e)(t−t _(φ))}dt=a ₀ cos Δφ,  (11A)(1/T)∫F _(VIR)(t)cos {f _(e)(t−t _(φ))}dt=a ₀ sin Δφ,  (11B)from which the quantities a₀ (≧0) and Δφ are easily determined. Thebaseline function b(t) is then determined fromb(t)=F _(VIR)(t)−a ₀{cos {(f _(c) −f _(e))(t−t _(φ))−Δφ}−a ₀{cos {(f_(c) +f _(e))(t−t _(φ))+Δφ}.  (12)The frequency difference (f_(c)−f_(e)) and frequency sum (f_(c)+f_(e))values are distinguished from each other in a normally functioning humanauditory system if the sum-frequency difference is at least equal to athreshold value, such as 250 Hz.

FIG. 5 is a flow chart illustrating a method for practicing theinvention. In step 51, at least one selected region of a visual image isrepresented by N selected visual image parameters, including at leastone of: vertical and horizontal location coordinates and time rate ofchange of location coordinate(s), relative to a signal recipient, of theregion; region predominant hue or wavelength, region brightness (averageand/or peak); and time rate of change of a visual signal parameter. Instep 52, the visual image region representatives are mapped onto Maudible signal attributes (M≧N), drawn from the following set ofattributes: carrier signal frequency; envelope signal frequency and timerate of change of envelope frequency; carrier signal-envelope signalphase difference at a selected time; baseline amplitude and time rate ofchange of baseline amplitude; envelope signal amplitude relative tobaseline amplitude; and signal time duration. In step 53, the audiblesignal attributes are presented sequentially in an audibly perceptiblemanner to the recipient. In an alternative step 53 (step 54), theaudible signal attributes are received and incorporated in one or moreaudible signals that is/are presented in an audibly perceptible mannerto a recipient.

The invention can be applied to provide audibly perceptible anddistinguishable signals, representing one or more selected regions of avisually perceptible image, for a sight-impaired person. Where more thanone VIR is represented, the audible signal representatives of the VIRsare preferably presented sequentially, with a small separation timeinterval (as little as a few tens of msec) between consecutiverepresentatives.

Where the visual image is a line drawing or other binary representation,the audible signal components can be configured to represent curvilinearand linear shapes, sizes and intersections. Where the visual imageprimarily represents interaction of dominant color masses in differentregions of the image, the dominant hues and shapes of these interactingregions can be represented audibly. For other reasons, anon-sight-impaired person may prefer to focus attention on attributes ofa region of an image that can be represented more accurately orintuitively by non-visual signals, for example, to extend the (visual)wavelength range of signals that can be perceived.

The invention can be applied to “enrich” image detail or manifest moreclearly some image details that are not evident where the region isviewed solely with reference to visible light wavelengths. For example,some details of a region may be hidden or muddled when viewed in visiblewavelength light but may become clear when the region is illuminatedwith, or viewed by, an instrument that is sensitive to, near-infraredlight (wavelength λ≈0.7-2 μm) or mid-infrared light (λ≈1-20 μm) orultraviolet light (λ≦0.4 μm). These hidden details can be converted toaudible signal parameter values that are more easily audibly perceivedas part of a received signal. Operated in this manner, the invention canseparately compensate for a relatively narrow (or relatively broad)visible wavelength sensitivity of the viewer and a relatively narrow (orrelatively broad) auditory frequency sensitivity of the same viewer orof a different viewer-recipient. Operated in this manner, the visiblewavelength sensitivity of a first (visual image) viewer of the imageregion can be adjusted and compensated for electronically by adjustingthe audible wavelength range of one or more of the audible signalparameters, before the transformed audible signal is received by thesame viewer or by a different viewer.

The invention can also be applied to provide audible signal componentsrepresenting shape signatures, sizes and estimated separation distancesfor objects that cannot be seen, or that are seen very imperfectly,because of signal interference, signal distortion and/or signalattenuation by the ambient environment. This may occur in a hazardousenvironment where fluids present provide an opaque, darkened ortranslucent view of objects in the environment, including moving ormotionless persons and objects that present a hazard

This interference may also occur in an airborne environment in whichrain, snow, hail, sleet, fog, condensation and/or other environmentalattributes prevent reasonably accurate visual perception of middledistance and far distance objects. A visual image region that is likelyto experience interference can be converted and presented as a sequenceof audio signal attributes that can be more easily or more accuratelyperceived or interpreted by an operator of an aircraft (airborne or onthe ground). The audio signal attributes may be extended to includeestimated closing velocity between the operator/aircraft and thenot-yet-seen object.

The invention can also be applied, in an environment of visual“confusion,” to focus on and provide information only on importantdetails among a clutter of unimportant details. The important detailsmay be characterized by certain parameters, and the system may focus oninitially-visual details that possess one or more of these parameters,converting the relevant information to audibly perceptible signals thatcontain this (converted) information. An example is a specified aircraftapproaching a destination airport surrounded by other airborne aircraft:the specified aircraft may wish to focus on and receive relevant(converted) audibly perceptible information for the immediatelypreceding aircraft and the immediately following aircraft in a queueformed by an air traffic controller to provide an orderly sequence oftouchdowns at the destination airport.

The invention can also be applied where visual signals representing theimage are more likely to experience signal interference, signaldistortion, signal attenuation and/or similar signal impairments thanare selected corresponding audible signals that represent certainparameters in these visual signals. The visual signals (now converted toaudible signals) may be transmitted through the ambient environment withreduced signal interference, reduced signal distortion and/or reducedsignal attenuation, and may be interpreted more accurately by a signalrecipient.

The invention can also be applied to provide an audible signalrepresenting P dimensions (P>2), formed or converted from atwo-dimensional visual image region. The audible signal may, forexample, provide depth clues, clues about a dominant hue or color orbrightness, if any, and clues about the maximum fineness of detailassociated with the image region, in addition to normal two-dimensionalinformation.

Consider a visual image region, such as a limited region of the image,and let p(t) represent an image region parameter that changes with time.The parameter may change continuously, or even differentiably, but inother more general situations p(t) may also change by a discrete amountat each of a sequence of spaced apart times {t_(n)}_(n), as{p(t_(n))}_(n), where p(t_(n))≠p(t_(n+1)), as discussed in thepreceding. Assuming that p(t_(n))+p(t₀)≠0 for n=1, 2, . . . one can forma normalized parameterq(t _(n))={p(t _(n))−p(t ₀)}/{p(t _(n))+p(t ₀)},  (13A)p(t _(n))/p(t ₀)={1−q(t _(n))}/{1+q(t _(n))},  (13B)which represents a difference or a ratio of a subsequent parametervalue, relative to an initial or preceding parameter value. Thisdifference or ratio can be represented audibly by a baseline signalamplitude difference (b(t_(n))−b(t₀)), amplitude ratio (b(t_(n))/b(t₀)),envelope frequency difference (f_(n)−f₀), or envelope frequency ratio(f_(n)/f₀), among other combinations.

As an example of application of the VISOR system, consider a battlefieldsituation in which one or more combatants, or one or more equipmentitems, are exposed to artillery shells or other projectiles, asillustrated in FIG. 6 and discussed in more detail in Appendices A, Band C.

In a first version, each combatant wears or carries a locationdetermination (“LD”) system, such as GPS, and is aware of thecombatant's present location coordinates within an accuracy of a fewmeters. A simple differential equation for a projectile ballistictrajectory is posited, and two or more observations, spaced apart intime, of the projectile location from each of two observers allowsestimation of the relevant shell trajectory parameters, includingprojectile launch point, projectile impact point and time, andprojectile explosive load, from which projectile injury and/orprojectile lethality regions can be estimated. Where the combatants areor may be within the disability or injury or lethality region for theprojectile, the combatants can be notified collectively of thisdevelopment by use of an audible (or visual) warning signal, such as asignal with monotonically decreasing (or increasing) frequency, with afinal frequency value f(end) that is near to or below a frequencycorresponding to disablement or injury or lethality. In this instance,each combatant receives a separate audible (or visual) warning signalwith monotonically varying frequency, having a final frequency f(end)that is specific for that combatant's present location on thebattlefield. That is, one combatant may be within an injury/lethalityregion, and another combatant may be outside this region, with aseparate audible (or visual) warning signal for each.

Where M (≧3) observations of projectile location are provided,projectile trajectory accuracy is enhanced by use of a statisticallyweighted average of trajectory location points. Distinction betweentrajectories of two or more projectiles that are present atsubstantially the same time is also available.

In a second version, the audible (or visual) warning signal has anundulatory signal frequency and/or an undulatory signal intensity, whichis different for a combatant location inside, as opposed to outside, aprobable disability or injury or lethality region relative to theestimated impact site.

In third and fourth versions, at least one (reference) combatant, butless than all the combatants, wears or carries an LD system, and theinjury or lethality region is estimated for the reference combatant.When the reference combatant is within the injury or lethality region,an appliance worn by the reference combatant issues an audible (orvisual) warning signal that is recognized by all nearby combatants.

Consider a battlefield situation in which one or more combatants, 61-i(i=1, 2, . . . ) are exposed to artillery shells or other projectiles,as illustrated in FIG. 6. Each combatant, 61-i wears or carries anappliance 63-i (i=1, 2, . . . ), including a receiver-processor for GPSsignals and/or other location determination (“LD”) signals, receivedfrom LD transmitters 65-j (j=1, . . . , J; J≧2) that are spaced apartfrom the combatants. The appliance 65-i associated with each combatant61-i is aware of the appliance location coordinates (xi, yi, zi) towithin an acceptably small inaccuracy. If differential GPS (“DGPS”)signals are used, the appliance location can be determined to within aninaccuracy of no more than one meter.

A projectile 62 is launched from a launch site location LS, spaced apartfrom the combatants 61-i, roughly targeting the combatants and followinga trajectory 67 that can be visually or (preferably) electromagneticallyobserved and estimated. For example, a trajectory estimation systemdisclosed in Appendices A, B and C, or any other system with acceptablepromptness of response, can be used for trajectory observation andestimation. A trajectory observation and estimation system 66 observesand provides a prompt, accurate estimation of the projectile trajectory67, including but not limited to an estimate of the impact site locationIS for the projectile 62. The location coordinates (x₆₇, y₆₇, z₆₇) forthe projectile impact location 67 are promptly transmitted to eachappliance 63-i, which promptly computes the separation distanced(i;sep)={xi−x ₆₇)²+(yi−y ₆₇)²+(zi−z ₆₇)²}^(1/2),  (14)and generates an audible, time varying signal S_(a)(t; i), illustratedgraphically in different versions in FIGS. 8A, 8B, 9A and 9B, that iscommunicated to the associated combatant 61-i. Optionally, eachcombatant 61-I receives a separately determined audible signal S_(a)(t;i) that is chosen or customized for that combatant's hearing system(including taking account of that combatant's hearing acuity or audiblesignal sensitivity versus frequency). In one version, the audible signalS_(a)(t; i) begins at a relatively high, but audible perceptiblefrequency f(0), and quickly and monotonically decreases to an endfrequency f(end) that is monotonically decreasing with decrease ofseparation distance d(i; sep) for the particular combatant 61-i.Optionally, the audible signal S_(a)(t; i) either terminates at the endfrequency f(end) or continues at that end frequency.

In a first version, the combatant 61-i or the appliance 63-I comparesthe end frequency f(end) with a frequency f(caution) (for which thecombatant has been trained) to determine if the estimated impactlocation 67 of the projectile is close enough (within a distance d(E),depending upon the estimated explosive load E, illustrated in FIG. 7) tothe combatant's own location to possibly cause death or serious injuryto an exposed combatant. If f(end)≦f(caution), the combatant quicklytakes defensive maneuvers, such as reducing exposure to the projectile'sexplosive force. If the estimated impact site location IS is furtheraway and f(end)>f(caution), the combatant may elect to take no defensivemaneuvers. Normally, a cautionary distance d(caution) varies inverselywith an estimate of the explosive load E carried by the projectile 62.

In a second version, the appliance 63-i numerically compares theestimated separation distance d(i; sep) with d(caution), computed forthe estimated explosive load of the projectile 62, and communicates theresult of this comparison audibly to the associated combatant 61-i,using an audible signal S_(a)(t; i) with monotonically varying frequencyf that decreases to f(end) (to be compared mentally with f(caution))according to the separation distance d(i; sep).

Alternatively, the audible signal frequency f(t) may increasemonotonically as d(i; impact) decreases so that f(end)≧f(caution) causesimplementation of defensive maneuvers by the combatant 61-i.

In the first version, each combatant 61-i receives a separatelydetermined audible signal S_(a)(t; i). In the second version, applicablewhere the relative locations of a group of combatants is substantiallyunchanging (the combatants remain in place or move as a group), a singleaudible signal S_(a)(t) can be provided, keyed to a separation distanced(ref; sep) of the estimated impact location from a reference combatant(real or virtual) for the group. In this version, the frequency rangef(end)≦f≦f(0) and the cautionary frequency f(caution) are preferablychosen to take account of the hearing acuities of each member in thegroup of combatants. Different versions of this example are discussed indetail in Appendices A, B and C.

As another example, consider a visual image region, a portion of alarger image, in which an object of interest moves toward the viewer oraway from the viewer at a substantial speed. Because of this movement,the apparent size (diameter viewed transverse to the direction of sight)of the object changes substantially with time. If the visual image isreduced in size and (re)defined so that this object is the dominantfeature of the (resulting visual image, change of the diameter with timecan be represented as an envelope frequency or baseline amplitude, forexample, with envelope frequency or baseline amplitude changing inproportion to the increase (or decrease) with time of the diameter ofthe object.

The system includes a “focus” mechanism (optional) that permits a visualimage region (part of a larger image) to be discretely or continuouslyreduced or increased or otherwise adjusted in size (horizontally andvertically, independent of each other) to redefine, and focus on, aselected smaller visual image region, in order to more clearly displaythe image temporal changes that are of most importance. This adjustmentin visual image region size can be implemented discretely by drawing aninitial quadrilateral or other polygon (rectangle, trapezoid, etc) as aborder around the region to which the visual image is restricted.Optionally, the resulting visual image region, thus redefined, maintainsits new shape, with an image diameter that increases or decreasesaccording to the change in diameter that occurs as a result ofdefinition of the selected smaller visual image region.

Appendix A Example of Projectile Trajectory Estimation

Where combatants are present on a battlefield and are exposed toartillery or armored vehicle fire, the injury and fatality count can bereduced if a probable impact point and a probable lethality radius forthe shell or other projectile can be quickly estimated and communicatedto the combatants. This requires some knowledge of the projectiletrajectory, preferably including probable launch point, probableexplosive load and probable impact point, as well as knowledge of otherflight variables.

The approach provides a procedure for quickly estimating relevantprojectile trajectory parameters and communicating this information in aformat that allows substantially instantaneous recognition of whether acombatant is inside or outside a probable lethality or injury ordisablement region for impact or explosion of the projectile. Thisinformation can be communicable to a group of adjacent combatants, toeach combatant individually, and/or to users of equipment items on abattlefield.

The approach is part of a system and method for (i) estimating relevantprojectile trajectory parameters from two or more temporally spacedprojectile observations from each of two or more spaced apart observers,(ii) estimating probable launch point coordinates, impact pointcoordinates, time of projectile impact and probableinjury/lethality/disablement region (for each of two ore morecombatants) from these parameters, (iii) use of (optional) subsequentobservations of projectile locations to improve the accuracy of impactprediction and to distinguish between trajectories of two or moreprojectiles that may be present at the same time.

In a first version, each combatant wears or carries a locationdetermination (“LD”) system, such as GPS, and is aware of thecombatant's present location coordinates within an accuracy of a fewmeters. A simple differential equation for a projectile ballistictrajectory is posited, and two or more observations, spaced apart intime, of the projectile location from each of two observers allowsestimation of the relevant shell trajectory parameters, includingprojectile launch point, projectile impact point and time, andprojectile explosive load, from which projectile injury and/orprojectile lethality regions can be estimated. Where the combatants areor may be within the disability or injury or lethality region for theprojectile, the combatants can be notified or warned collectively ofthis development by use of an audible (or visual) warning signal, suchas a signal with monotonically decreasing (or increasing) frequency,with a final frequency value f(end) that is near to or below a frequencycorresponding to disability or injury or lethality. In this instance,each combatant receives a separate audible (or visual) warning signalwith monotonically varying frequency, having a final frequency f(end)that is specific for that combatant's present location on thebattlefield. That is, one combatant may be within an injury/lethalityregion, and another combatant may be outside this region, with aseparate audible (or visual) warning signal for each.

Where M (≧3) observations of projectile location are provided,projectile trajectory accuracy is enhanced by use of a statisticallyweighted average of trajectory location points. Distinction betweentrajectories of two or more projectiles that are present atsubstantially the same time is also available.

In a second version, the audible (or visual) warning signal has anundulatory signal frequency and/or an undulatory signal intensity, whichis different for a combatant location inside, as opposed to outside, aprobable disability or injury or lethality region relative to theestimated impact site.

In third and fourth versions, at least one (reference) combatant, butless than all the combatants, wears or carries an LD system, and theinjury or lethality region is estimated for the reference combatant.When the reference combatant is within the injury or lethality region,an appliance worn by the reference combatant issues an audible (orvisual) warning signal that is recognized by all nearby combatants.

FIG. 6 illustrates a general environment, with two or more combatants,61-n (n=1, . . . , N; N≧2), present and exposed to artillery shells orother projectiles 62 that are directed against the combatants. In oneembodiment, each combatant 61-n wears or carries a receiver-processor63-n, including an antenna 64-n, that receives location determination(“LD”) signals from Q spaced apart LD signal sources 65-q (q=1, . . . ,Q; Q≧3) and estimates the present location coordinates, r_(n)=(x_(n),y_(n), z_(n)) of the combatant 61-n or of the antenna 64-n. Thereceiver/processor 63-n also receives trajectory parameter information(“TPI”) from one or more TPI sources 66-p (p=1, . . . , P; P≧1) thattrack and report TPI for one or more visible trajectories 67corresponding to at least one projectile 62. Determination or estimationof trajectory information from two or more observers at two or morespaced apart observation times is presented in Appendix C.

It is assumed here that the TPI signals for a given trajectory 62 arereceived by a receiver-processor 63-n, including information thatpermits estimation of at least one of (i) launch site coordinates,r_(L)=(x_(L), y_(L), z_(L)), (ii) impact site coordinates, r_(I)=(x_(I),y_(I), z_(L)), and (iii) probable explosive load E of the projectile.The receiver-processor 63-n computes a separation distanced(n;sep)={(x _(I) −x _(n))²+(y _(I) −y _(n))²+(z _(I) −z_(n))²}^(1/2),  (A-1)and compares this separation distance with a probable impact effectdistance, d(E)=d(E; injury or d(E; death), illustrated graphically inFIG. 7.

If the Conditiond(n;sep)≦d(E)  (A-2)is satisfied so that injury or death from the projectile (which has notyet reached the impact site) is probable, the receiver-processor 63-ngenerates a first audible (or visual) warning signal s_(a)(t; n; 1) witha monotonically decreasing frequency, f=i_(i)(t; n; decr) (FIG. 8A) ormonotone increasing, f=f_(i)(t; n; incr) (FIG. 8B). The distance d(E)may be different in different directions, measured from the estimatedimpact site, and may be different for each combatant, based on theprotective gear worn by each combatant If the signal frequency f_(i)(t;n; decr; 1) is monotone decreasing (FIG. 8A) and decreases below(preferably substantially below) a known cautionary frequency,f=f(caut), the corresponding combatant 61-n is made aware that he/she islikely to suffer injury or death from explosion or impact of theincoming projectile, and the combatant should quickly seek cover orprotection, if possible. If the signal frequency, f=f_(i)(t; n; incr;1), is monotone increasing (FIG. 8B) and increases above (preferablysubstantially above) a known cautionary frequency, f=f(caut), thecorresponding combatant 61-n is made aware that he/she is likely tosuffer injury or death from explosion or impact of the incomingprojectile, and the combatant should seek cover or protection, ifpossible. If d(n; sep)>d(E) so that injury or death from explosion ofthe incoming projectile is less likely or unlikely, the audible (orvisual) signal frequency, f=f_(i)(t; n; decr; 1) will stop decreasing atan end frequency, f(end)>f(caut), and the audible (or visual) signalfrequency, f=f_(i)(t; n; incr) will stop increasing at an end frequency,f(end)<f(caut), as illustrated in FIGS. 8A and 8B, respectively.

In this first version, the location of each combatant 61-n is known, towithin an inaccuracy of a few meters or less, using an LD device that ispart of the receiver-processor 63-n, and a separation distance d(n; sep)is calculated for each combatant 61-n, preferably using informationreceived at and/or computed by the corresponding receiver-processor63-n. The condition in Eq. (A-2) is tested separately for each combatant61-n to determine if this condition is satisfied. For each combatant61-n for which the condition (A-2) is satisfied, the receiver-processor63-n generates a first audible (or visual) warning signal s_(a)(t; n;decr) in which the monotone decreasing frequency f_(i)(t; n; decr)decreases to substantially below the cautionary frequency f(caut); or,alternatively, the monotone increasing frequency f_(i)(t; n; incr)increases to substantially above the cautionary frequency f(caut). Wherethe condition (A-2) is not satisfied for a particular combatant 61-n,the monotone decreasing frequency f_(i)(t; n; decr) terminates orplateaus at an end frequency f(end)>f(caut); and the monotone increasingfrequency f_(i)(t; n; incr) terminates or plateaus at an end frequencyf(end)<f(caut). Optionally, the cautionary frequency f(caut) can bechosen separately for each combatant, to approximately coincide with afrequency of maximum audible (or visual) sensitivity for that combatant,to compensate for differences in frequency sensitivity betweencombatants.

In a second version, where the condition (A-2) is satisfied for acombatant 61-n, the system issues a second audible (or visual) warningsignal s_(a)(t; n; 2), which has (i) a rapidly varying, preferablyundulatory, frequency f_(i)(t; n; var) and/or (ii) a rapidly varying,preferably undulatory, signal intensity I_(i)(t; n; var), as illustratedin FIGS. 9A and 9B, to make the combatant aware of this condition. Wherethe condition (A-2) is not satisfied, the system issues adistinguishable audible (or visual) warning signal, for example, aconstant frequency and/or constant intensity signal. In the first andsecond embodiments, the signal intensity of the output signal s_(i,n)(t)is preferably low enough, and/or the angular distribution of the signalintensity is sufficiently narrow, so that this warning signal is audible(or visible) only to an individual combatant 61-n.

In a third version, the receiver-processor 63-n 1 worn or carried by atleast one combatant 61-n 1 does not include an LD system so that thiscombatant and the associated receiver-processor 63-n 1 are not aware ofthe corresponding location coordinates of this combatant. In thissituation, it is preferable that at least one referencereceiver-processor 63-n 2 includes an LD system so that thecorresponding reference combatant 61-n 2 and the associated referencereceiver-processor 63-n 2 are aware of the corresponding locationcoordinates for this reference combatant. The receiver-processor 63-n 2receives the TPI signals from the TPI source(s) 66-p and determines theseparation distance d(n; sep) for itself as in Eq. (A-1). If thecondition (A-2) is satisfied, with n=n2, the receiver-processor 63-n 2generates the audible (or visual) first warning signal s_(a)(t; n; decr;1), which is loud or intense enough to be recognized by any combatantwithin a selected distance D (e.g., D=50-100 meters) from the locationof the reference combatant 63-n 2. As in the first embodiment, the firstaudible (or visual) warning signal s_(i)(t; n; decr or incr) has amonotonically decreasing frequency, f=f_(Ii)(t; n; decr) (FIG. 8A) or amonotone increasing frequency, f=f_(i)(t; n; incr) (FIG. 8B), and thissignal is recognized by all combatants within the distance D from thereceiver-processor 63-n 2. The procedure for the remainder of the thirdembodiment proceeds as in the first embodiment.

In a fourth version, at least one, but less than all, of thereceiver-processors 63-n 2 includes an LD system, and if d(n; sep)≦d(E),the system issues a second audible (or visual) warning signal s_(a)(t;n; 2) with (i) a rapidly varying, preferably undulatory, frequencyf_(a)(t; n; var) and/or (ii) a rapidly varying, preferably undulatory,signal intensity I_(i)(t; n; var), as illustrated in FIGS. 9A and 9B.The remainder of the fourth embodiment proceeds as in the secondembodiment.

In each of these versions, a receiver-processor 63-n provides a firstaudible (or visual) warning signal s_(a)(t; n; decr or incr; 1) if thecondition (A-2) is satisfied and provides a different, distinguishablewarning signal if the condition (A-2) is not satisfied. Two or moredistinguishable audible (or visual) warning signals, rather thantext-based, symbol-based or color-based (visual) signals, are preferablyused here, because an aural system is believed to be more flexible, tooffer greater discrimination, and to offer greater range, than does avisual system. The launch site LS may be ground-based or may be part ofan airborne or mobile vehicle.

The invention can also be used to estimate an impact effect distance,d(E)=d(E; disablement), for disablement of an equipment item, such as avehicle, a weapon or an observation instrument, that is located in ageographical region adjacent to the impact site, as graphicallyillustrated in FIG. 7. Alternatively, a visually perceptible or audiblyperceptible warning signal may be issued to indicate that a separationdistance d(sep) is no greater than an impact effect distance d(E) for(i) death of a combatant, (ii) serious injury to a combatant, or (iii)disablement of part or all of an equipment item, preferably with aperceptibly different signal being used for each of the categories (i),(ii) and (iii). The different impact effect distances for these threecategories should be determined experimentally, at least in part.

Appendix B Trajectory Equations

Consider a projectile, such as an artillery shell, that is launched froma launch site with initially-unknown launch site coordinates (x_(L),y_(L), z_(L)), travels through an atmosphere, and is subject only toatmospheric and gravitational (g) forces, as illustrated in FIG. 6. Theprojectile 62 has a mass of m and an initial launch velocity v0immediately after launch, both initially unknown. A suitable secondorder differential equation describing ballistic motion of theprojectile after launch from a launch site LS ism(d ² r/dt ²)=F _(w) −kmg,  (B-1)where F_(w) is an estimated wind force vector (assumed constant),dependent upon the shape parameters and other relevant details of theprojectile but not directly upon the mass m, and r(t) is location vectorof the projectile at time t.

Where the wind force is negligible or is ignored, a suitable solution ofEq. (B-1) isr(t)={(F _(w) /m)−kg}(t−t0)²/2+b(t−t0)+c,  (B-2)b=b(i1,i2)={r(t _(i2))−r(t _(i1))}/(t _(i2) −t _(i1)){(F _(w) /m)−kg}(t_(i1) +t _(i2)−2t ₀),  (B-3)c=c(i1,i2)={r(t _(i1))+r(t _(i2))+{(F _(w) /m)−k g}{(t _(i2) −t₀)²}}/2,  (B-4)where r(t=t_(i1)) and r(t=t_(i2)) are two observations of locationcoordinates for the projectile at distinct times t=t_(i1) and t=t_(i2)(I=1, 2, . . . ), and t₀ is a selected but arbitrary time value (e.g.,t₀=(t_(i1)+t_(i2))/2). Where the observation time values t_(i1) andt_(i2) are known, the solution r(t) can be extended backward and forwardin time to estimate a launch time, t=t_(L), a corresponding launch siteLS, an impact time, t=t_(I), and a corresponding impact site IS forwhichr(t_(L))εS1,  (B-5a)r(t_(I))εS2,  (B-5b)where S_(L) is a known launch surface and S_(I) is a known impactsurface (e.g., planar or spheroidal). The launch time, t=t_(L), theimpact time, t=t_(I), the launch site coordinates (x_(L), y_(L), z_(L)),and the impact site coordinates (x_(I), y_(I), z_(I)), are estimatedfrom Eqs. (B-2), (B-3) and (B-4).

Two or more projectile observations (at times t=t_(i1) and t=t_(i2) witht_(i1)<t_(i2)) from each of two or more spaced apart observers are usedto provide trajectory vector values r(t_(i1)) and r(t_(i2)). Whereprojectile observations are available at J>2 distinct times,t=t_(ij)(j=1, . . . , J≧3), with t_(i1)<t_(i2)< . . . t_(iJ), one canobtain a potentially more accurate estimation of the trajectory byreplacing the trajectory parameters b(i1,i2) and c(i1,i2) with filteredor statistically weighted parameter values, b^ and c^, respectively,defined by

$\begin{matrix}{{b^{\bigwedge} = {\sum\limits_{j = 1}^{J}\;{\sum\limits_{j^{\prime} = {j + 1}}^{J}\;{{h\left( {{ij},{ij}^{\prime}} \right)}{b\left( {{ij},{ij}^{\prime}} \right)}}}}},} & \left( {B - 6} \right) \\{c^{\bigwedge} = {\sum\limits_{j = 1}^{J}\;{\sum\limits_{j^{\prime} = {j + 1}}^{J}\;{{h\left( {{ij},{ij}^{\prime}} \right)}{c\left( {{ij},{ij}^{\prime}} \right)}}}}} & \left( {B - 7} \right)\end{matrix}$where h(ij,ij′) are normalized, non-negative filter weights satisfyingh(ij,ij′)≧0(j′<j),  (B-8)Σh(ij,ij′)=1  (B-8)

j<j′

The projectile launch velocity

$\begin{matrix}{{\left( {{\mathbb{d}r}/{\mathbb{d}t}} \right)_{t = {tL}} = {v_{0} = {{\left\{ {{F_{w}/m} - {kg}} \right\}\left( {t_{L} - {t\; 0}} \right)} + b}}},} & \left( {B - 10} \right)\end{matrix}$determined immediately after launch, may be computed and used toestimate the projectile explosive load E, relying in part upon adatabase of launch velocity v₀ for each of the different projectiles inthe adversary's arsenal. The projectile explosive load E plus areference curve or database of separation distance d(E) for seriousinjury or lethality (FIGS. 8A and 8B) is used to estimate whether acombatant is within a probable injury or lethality region for theestimated impact site.

Where the value of F_(w) is known (e.g., from local wind observations),the projectile mass m can be estimated. By consulting an appropriatedatabase of the adversary's projectiles and comparing the projectilelaunch velocity, determined immediately after launch, plus theprojectile mass m, an estimate of projectile explosive load E can bemade. Using the information contained in FIG. 7 for the particularprojectile used, an impact effect distance d(E) corresponding to seriousinjury or lethality or equipment disablement can be estimated and usedin the preceding development in connection with Eq. (A-2) to determinewhich audible (or visual) warning signal s_(a,n)(t) should be providedfor a particular combatant, or for all combatants. in a given region.Two or more different warning signals may be provided, corresponding todifferent dangers from projectile impact.

Appendix C Trajectory Observations

FIG. 10 illustrates a projectile that is observed at substantially thesame time at each of two (or more) known observation locations, O1 andO2, having the respective location coordinates (x₁, y₁, z₁) and (x₂, y₂,z₂). Each of these observers observes a projectile P with unknownlocation coordinates (x_(p), y_(p), z_(p)) at substantially the sametime, through measurements of a separation distance d_(m)(m=1, 2) andspherical coordinates (φ_(m),θ_(m)), measured relative to a line ofsight segment L_(m) from Om to P. The observer locations, O1 and O2 andpresent projectile location P have coordinates referenced to anarbitrary but fixed coordinate system, including a plane Π determined byline segments O-O1 and O-O2.

A directly measured separation distance d_(m) can be determined bymeasuring a round trip return time of a radar or other electromagnetic,electronic or acoustic signal, issued at Om and received as a reflectedreturn signal a time Δt_(m) later,d _(m)(meas)=c(Δt _(m))/2(m=1,2)  (C-1)where c is a velocity of signal propagation in the ambient medium.

The line of sight segment L_(m) from Om to P is described parametricallyas follows for each observation point:x=d cos θ_(m) cos φ_(m) +x _(m),  (C-2a)y=d cos θ_(m) sin φ_(m) +y _(m),  (C-2b)z=d sin θ_(m) +z _(m),  (C-2c)where (θ,φ) are spherical coordinates, referenced to the same coordinatesystem, and d is the distance along the line of sight L_(m). From theEqs. (C-2), one recovers consistency relations,(x−x _(m))²+(y−y _(m))²+(z−z _(m))² =d ²{cos²θ_(m) cos²φ_(m)+cos²θ_(m)sin²φ_(m)+sin²θ_(m)}=d²  (C-3)Equations (C-2a) and (C-2b) can be re-expressed as(x−x _(m)/{cos θ_(m) cos φ_(m)}=(y−y _(m))/{cos θ_(m) sin φ_(m)}=(z−z_(m))/{sin θ_(m) }=d _(m).  (C-4)

The location (x_(P), y_(P), z_(P)) is the unique intersection of theline of sight segments L₁ and L₂. From Eqs. (C-2a) and (C-2b) one infersthat

$\begin{matrix}{{{\begin{matrix}d_{1} \\d_{2}\end{matrix}}\begin{matrix}{= \left\{ {\det(M)} \right\}^{- 1}} \\\;\end{matrix}{\begin{matrix}{{- \cos}\;\theta_{2}\sin\;\phi_{2}} & {\cos\;\theta_{2}\cos\;\phi_{2}} \\{{- \cos}\;\theta_{1}\sin\;\phi_{1}} & {\cos\;\theta_{1}\cos\;\phi_{1}}\end{matrix}}{\begin{matrix}{x_{2} - x_{1}} \\{y_{2} - y_{1}}\end{matrix}}},} & \left( {C - 5} \right)\end{matrix}$det(M)=cos η₂ cos η₂ sin(φ₁−φ₂)  (C-6)x _(P) =x ₁ +d ₁ cos θ₁ cos θ₁ cos φ₁ =x ₂ +d ₂ cos θ₂ cos φ₂,  (C-7)y _(P) =y ₁ +d ₁ cos θ₁ sin φ₁ =y ₂ +d ₂ cos θ₂ sin φ₂,  (C-8)z _(P) =z ₁ +d ₁ sin θ₁ =z ₂ +d ₂ sin θ₂,  (C-9)from which the present location coordinates (x_(P), y_(P), z_(P)) can beestimated.

FIG. 11 illustrates a situation in which two (or more) distinctprojectiles, 62-1 and 62-2, following different trajectories, 67-1 and67-2, are present at the same time so that trajectory confusion ispossible. For convenience, it is assumed that two observation sites,numbers m and m′, are paired with each other for estimation of thepresent values of the projectile coordinates (x_(P), y_(P), z_(P)) andthat transmission of the interrogation signals from each of these twopaired sites are coordinated, preferably occurring at substantially thesame time.

As a first approach to suppressing the possibility of confusion, each(first) observation site may use a different interrogation frequencyf1(interr) and f2(interr), such as a distinguishable radar returnfrequency, and each observation site (m=1, 2) filters the returnsignal(s), discussed in the preceding, to identify its own return signaland to determine the observation site-projectile separation distanced_(OmP) for itself. Where the first observation site (m) is also awareof the interrogation frequency f_(m′)(interr) used by the secondobservation site (number m′≠m) to cooperatively determine the presentlocation coordinates (x_(P), y_(P), z_(P)) of the projectile, asdiscussed in the preceding, the elapsed time Δt_(m′) and separationdistance d_(Om′P) associated with a return signal for the second sitecan be estimated at the first site asΔt _(m′)(est)=Δt _(m)+(t _(m′) −t _(m)),  (C-10)d _(m′)(est)=c(Δt _(m′))/2,  (C-11)where t_(m) and t_(m′) are the measured absolute times for receipt, atthe first site, of the return signals with the associated returnfrequencies f1(interr) and f2(interr), respectively. The estimatedseparation distances, d_(m)(est) and the directly measured separationdistance d_(m)(meas), for the two paired observation sites can becompared with each other (for m=1 and, separately, for m=2) to confirmor refute the hypothesis that each of the two paired sites is observingthe same projectile. If each of the two paired sites is observing adifferent projectile, at least one of the two pairs of comparedseparation times, Δt_(m)(est) and Δt_(m)(meas), should be in substantialdisagreement with the other corresponding separation time.

This approach relies upon (1) filtering the return signals received inorder to distinguish between the return signal frequencies f1(interr),f2(interr) and any other return signal frequency (2) use of separatereturn signal gating, at the first site and/or at the second site, ofthe return signals for the first and second sites.

In a second approach, the projectile coordinates (x_(P), y_(P),z_(P))_(t=ti) are determined from observations at times t=t_(i) (i=1, 2,3, 4) with t₁<t₂<<t₃<t₄. The projectile trajectory parameters fort_(i)=t₁ and t=t₂ are compared with the corresponding projectiletrajectory parameters for t_(i)=t₃ and t=t₄. If the correspondingparameters are sufficiently close to each other, this tends to confirmthe hypothesis that each of the two paired sites is observing the sameprojectile. If the two sets of corresponding trajectory parameters aremarkedly different from each other, this indicates that each of thefirst and second observation sites is likely observing a differentprojectile.

1. A method for enhancing or extending a vision system of a human being,the method comprising providing a computer that is programmed: torepresent at least one selected visual image region, having at least onecolor or hue associated with the region, in terms of at least one visualimage parameter for the region, the at least one visual image parameterincluding at least one of: vertical coordinate range of the region;horizontal coordinate range of the region; center coordinates for theregion; region brightness; dominant hue range or wavelength range of theregion; change in a parameter value that characterizes the visual image,with respect to a reference parameter value; and time rate of change TRCin a parameter value that helps characterize the visual image region; toassociate each of the visual image parameters with at least one of afirst set of M1 audio signal attributes (M1≧1), the first set ofattributes being drawn from signal carrier frequency, signal envelopefrequency, carrier signal-envelope signal phase difference at a selectedtime, baseline amplitude, envelope signal amplitude relative to baselineamplitude, and signal time duration, and with a second set of at leastone of M2 audio signal attributes (M2≧1), the second set of attributesbeing drawn from time rate of change of envelope frequency andtime-rate-of-change of baseline amplitude; to allow the selected visualimage region to be adjusted in at least one of a horizontal dimensionand a vertical dimension so that the visual image region primarilyincludes a selected object whose apparent diameter is increasing withtime or is decreasing with time; and to present the M1+M2 audio signalattributes sequentially or simultaneously for the at least one selectedregion.
 2. The method of claim 1, wherein said computer is furtherprogrammed to provide a presentation format in which at least one ofsaid signal carrier frequency, said signal envelope frequency, saidsignal carrier-signal envelope phase difference, said baselineamplitude, said time-rate-of-change of said baseline amplitude, saidenvelope signal amplitude relative to said baseline amplitude, and saidsignal time duration changes monotonically with at least one of acoordinate representing said vertical location of said visual imageregion, a coordinate representing said horizontal location of saidcorresponding visual image region, one or more coordinates representinga range of horizontal coordinates of said corresponding visual imageregion, one or more coordinates representing a range of verticalcoordinates of said corresponding visual image region, a coordinaterepresenting brightness of said corresponding visual image component,and a coordinate representing said dominant wavelength or dominant hueof said corresponding visual image region.
 3. The method of claim 2,wherein said computer is further programmed to choose said monotonicchange to be at least one of (a) a logarithmic change, (b) a power lawchange and (c) a linear change.
 4. The method of claim 2, wherein saidcomputer is further programmed to choose said monotonic change to be amonotonically increasing change.
 5. The method of claim 2, wherein saidcomputer is further programmed to choose said monotonic change to be amonotonically decreasing change.
 6. The method of claim 1, wherein saidcomputer is further programmed: to determine said dominant hue orwavelength by a process comprising determining at least one selectedwavelength range, drawn from wavelength ranges in ultraviolet, visible,near infrared and mid-infrared wavelengths, that contains said dominanthue or wavelength for said visual image component; and to associate saidat least one of said audio signal attributes with the selectedwavelength range.
 7. The method of claim 1, further comprising providingsaid at least one of said four audio signal attributes for a selectedrecipient whose sight is impaired.
 8. The method of claim 1, furthercomprising providing at least one of said signal carrier frequency andsaid signal envelope frequency in a frequency range determined byhearing acuity of a selected recipient.
 9. The method of claim 1,further comprising choosing said at least one visual image component tohave at least one selected wavelength that is part of at least one of anultraviolet spectrum, a near-infrared spectrum and a mid-infraredspectrum.
 10. The method of claim 1, further comprising representing, toa selected recipient having an auditory communication system, said atleast one of said audio signal attributes with at least one frequency towhich the selected recipient's auditory communication system issensitive.
 11. The method of claim 1, further comprising providing saidvisual image component for said region that cannot be accuratelyperceived visually because of at least one of signal interference,signal distortion and signal attenuation by an ambient environment. 12.The method of claim 11, wherein said ambient environment includes atleast one of rain, snow, ice, hail, sleet, fog, condensation, lightning,and transition between daylight and nighttime.
 13. The method of claim1, further comprising choosing said at least one of said audio signalattributes so that said at least one audio signal attribute can betransmitted through an environment with at least one of reduced signalinterference, reduced signal distortion and reduced signal attenuation.14. The method of claim 1, further comprising analyzing said change insaid parameter that characterizes said visual image.
 15. The method ofclaim 1, further comprising: identifying at least one of said time rateof change TRC of said parameter value that helps characterize saidregion; and representing said time rate of change TRC of said parametervalue as at least one of: (i) a continuous change with time, (ii) adiscrete change at two or more spaced apart times, and (iii) a discretechange, only when a magnitude of a difference between a first value ofsaid parameter and a second value of said parameter is at least equal toa selected threshold magnitude.
 16. A system for enhancing or extendinga vision system of a human being, the system comprising providing acomputer that is programmed: to represent at least one selected visualimage region, having at least one color or hue associated with theregion, in terms of at least one visual image parameter for the region,the at least one visual image parameter including at least one of:vertical coordinate range of the region; horizontal coordinate range ofthe region; center coordinates for the region; region brightness;dominant hue range or wavelength range of the region; change in aparameter value that characterizes the visual image, with respect to areference parameter value; and time rate of change TRC in a parametervalue that helps characterize the visual image region; to associate eachof the visual image parameters with at least one of a first set of M1audio signal attributes (M1≧1), the first set of attributes being drawnfrom signal carrier frequency, signal envelope frequency, carriersignal-envelope signal phase difference at a selected time, baselineamplitude, envelope signal amplitude relative to baseline amplitude, andsignal time duration, and with a second set of at least one of M2 audiosignal attributes (M2≧1), the second set of attributes being drawn fromtime rate of change of envelope frequency and time-rate-of-change ofbaseline amplitude; and to analyze an object, within the visual imageregion, that is moving toward a viewer or away from a viewer of thevisual image region so that a lateral diameter of the object isincreasing or is decreasing, respectively, and to transmit at least oneof said M1+M2 audio signal attributes so that at least one of a signalfrequency and a signal amplitude is increasing or decreasing in time ata rate corresponding to the increase or decrease in the lateral diameterwith time; and to present the M1+M2 audio signal attributes sequentiallyor simultaneously for the at least one selected region.
 17. The systemof claim 16, wherein said computer is further programmed to provide apresentation format in which at least one of said signal carrierfrequency, said signal envelope frequency, said signal carrier-signalenvelope phase difference, said baseline amplitude, saidtime-rate-of-change of said baseline amplitude, said envelope signalamplitude relative to said baseline amplitude, and said signal timeduration changes monotonically with at least one of a coordinaterepresenting said vertical location of said visual image region, acoordinate representing said horizontal location of said correspondingvisual image region, one or more coordinates representing a range ofhorizontal coordinates of said corresponding visual image region, one ormore coordinates representing a range of vertical coordinates of saidcorresponding visual image region, a coordinate representing brightnessof said corresponding visual image component, and a coordinaterepresenting said dominant wavelength or dominant hue of saidcorresponding visual image region.
 18. The system of claim 17, whereinsaid computer is further programmed to choose said monotonic change tobe at least one of (a) a logarithmic change, (b) a power law change and(c) a linear change.
 19. The system of claim 17, wherein said computeris further programmed to choose said monotonic change to be amonotonically increasing change.
 20. The system of claim 17, whereinsaid computer is further programmed to choose said monotonic change tobe a monotonically decreasing change.
 21. The system of claim 16,wherein said computer is further programmed: to determine said dominanthue or wavelength by a process comprising determining at least oneselected wavelength range, drawn from wavelength ranges in ultraviolet,visible, near infrared and mid-infrared wavelengths, that contains saiddominant hue or wavelength for said visual image component; and toassociate said at least one of said audio signal attributes with theselected wavelength range.
 22. The system of claim 16, wherein said atleast one of said four audio signal attributes is provided for aselected recipient whose sight is impaired.
 23. The system of claim 16,wherein at least one of said signal carrier frequency and said signalenvelope frequency is provided in a frequency range determined byhearing acuity of a selected recipient.
 24. The system of claim 16,wherein said at least one visual image component has at least oneselected wavelength that is part of at least one of an ultravioletspectrum, a near-infrared spectrum and a mid-infrared spectrum.
 25. Thesystem of claim 16, further comprising an auditory communication systemthat represents, to a selected recipient, said at least one of saidaudio signal attributes with at least one frequency to which theselected recipient's auditory communication system is sensitive.
 26. Thesystem of claim 16, wherein said visual image component for said regioncannot be accurately perceived visually because of at least one ofsignal interference, signal distortion and signal attenuation by anambient environment.
 27. The system of claim 26, wherein said ambientenvironment includes at least one of rain, snow, ice, hail, sleet, fog,condensation, lightning, and transition between daylight and nighttime.28. The system of claim 16, wherein said at least one of said audiosignal attributes is chosen so that said at least one audio signalattribute can be transmitted through an environment with at least one ofreduced signal interference, reduced signal distortion and reducedsignal attenuation.
 29. The system of claim 16, wherein said computer isfurther programmed to analyze said change in said parameter thatcharacterizes said visual image.
 30. The system of claim 16, whereinsaid computer is further programmed: to identify at least one of saidtime rate of change TRC of said parameter value that helps characterizesaid region; and to represent said time rate of change TRC of saidparameter value as at least one of: (i) a continuous change with time,(ii) a discrete change at two or more spaced apart times, and (iii) adiscrete change, only when a magnitude of a difference between a firstvalue of said parameter and a second value of said parameter is at leastequal to a selected threshold magnitude.
 31. A method for enhancing orextending a vision system of a human being, the method comprisingproviding a computer that is programmed: to represent at least oneselected visual image region, having at least one color or hueassociated with the region, in terms of at least one visual imageparameter for the region, the at least one visual image parameterincluding at least one of: vertical coordinate range of the region;horizontal coordinate range of the region; center coordinates for theregion; region brightness; dominant hue range or wavelength range of theregion; change in a parameter value that characterizes the visual image,with respect to a reference parameter value; and time rate of change TRCin a parameter value that helps characterize the visual image region; toassociate each of the visual image parameters with at least one of afirst set of M1 audio signal attributes (M1≧1), the first set ofattributes being drawn from signal carrier frequency, signal envelopefrequency, carrier signal-envelope signal phase difference at a selectedtime, baseline amplitude, envelope signal amplitude relative to baselineamplitude, and signal time duration, and with a second set of at leastone of M2 audio signal attributes (M2≧1), the second set of attributesbeing drawn from time rate of change of envelope frequency andtime-rate-of-change of baseline amplitude; to analyze an object, withinthe visual image region, that is moving toward a viewer or away from aviewer of the visual image region so that a lateral diameter of theobject is increasing or is decreasing, respectively, and to transmit atleast one of said M1+M2 audio signal attributes so that at least one ofa signal frequency and a signal amplitude is increasing or decreasing intime at a rate corresponding to the increase or decrease in the lateraldiameter with time; and to present the M1+M2 audio signal attributessequentially or simultaneously for the at least one selected region. 32.A system for enhancing or extending a vision system of a human being,the system comprising providing a computer that is programmed: torepresent at least one selected visual image region, having at least onecolor or hue associated with the region, in terms of at least one visualimage parameter for the region, the at least one visual image parameterincluding at least one of: vertical coordinate range of the region;horizontal coordinate range of the region; center coordinates for theregion; region brightness; dominant hue range or wavelength range of theregion; change in a parameter value that characterizes the visual image,with respect to a reference parameter value; and time rate of change TRCin a parameter value that helps characterize the visual image region; toassociate each of the visual image parameters with at least one of afirst set of M1 audio signal attributes (M1≧1), the first set ofattributes being drawn from signal carrier frequency, signal envelopefrequency, carrier signal-envelope signal phase difference at a selectedtime, baseline amplitude, envelope signal amplitude relative to baselineamplitude, and signal time duration, and with a second set of at leastone of M2 audio signal attributes (M2≧1), the second set of attributesbeing drawn from time rate of change of envelope frequency andtime-rate-of-change of baseline amplitude; to allow the selected visualimage region to be reduced in at least one of a horizontal dimension anda vertical dimension so that the visual image region primarily includesa selected object whose apparent diameter is increasing with time or isdecreasing with time; and to present the M1+M2 audio signal attributessequentially or simultaneously for the at least one selected region. 33.A method for enhancing or extending a vision system of a human being,the method comprising providing a computer that is programmed: torepresent at least one selected visual image region, having at least onecolor or hue associated with the region, in terms of at least one visualimage parameter for the region, the at least one visual image parameterincluding at least one of: vertical coordinate range of the region;horizontal coordinate range, of the region; center coordinates for theregion; region brightness; dominant hue range or wavelength range of theregion; change in a parameter value that characterizes the visual image,with respect to a reference parameter value; and time rate of change TRCin a parameter value that helps characterize the visual image region; toassociate each of the visual image parameters with at least one of afirst set of M1 audio signal attributes (M1≧1), the first set ofattributes being drawn from signal carrier frequency, signal envelopefrequency, carrier signal-envelope signal phase difference at a selectedtime, baseline amplitude, envelope signal amplitude relative to baselineamplitude, and signal time duration, and with a second set of at leastone of M2 audio signal attributes (M2≧1), the second set of attributesbeing drawn from time rate of change of envelope frequency andtime-rate-of-change of baseline amplitude; to present the M1+M2 audiosignal attributes sequentially or simultaneously for the at least oneselected region as an audibly perceptible signal (APS) that isrepresentable as an equationS _(a)(t)=b(t)+a(t)sin {ƒ_(e)(t)t+Φ _(e)} sin {ƒ_(c) t+Φ _(c)} whereb(t) is a baseline amplitude for the APS, expressed as a function oftime t and having a time rate of change db/dt associated with thebaseline amplitude, a(t) is an envelope signal amplitude, ƒ_(e)(t) is asignal envelope frequency, which may vary with the time t, φ_(e) is asignal envelope phase, ƒ_(c) is a carrier signal frequency, and Φ_(c) acarrier signal phase; and to analyze an object, within the visual imageregion, that is moving toward a viewer or away from a viewer of thevisual image region so that a lateral diameter of the object isincreasing or is decreasing, respectively, and to transmit at least oneof said M1+M2 audio signal attributes so that at least one of a signalfrequency, ƒ_(c)(t) or ƒ_(c), and a signal amplitude, a(t) or b(t), isincreasing or decreasing in time at a rate corresponding to the increaseor decrease in diameter with time.